CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer

Camera and radar sensors have significant advantages in cost, reliability, maintenance compared to LiDAR. Existing fusion methods often fuse the outputs of single modalities at result-level, called late strategy. This can benefit from using off-the-shelf sensor detection algorithms, but cannot fully exploit complementary properties sensors, thus having limited performance despite huge potential camera-radar fusion. Here we propose a novel proposal-level early approach that effectively exploits both spatial contextual camera for 3D object detection. Our framework first associates image proposal with points polar coordinate system efficiently handle discrepancy between properties. Using this as stage, following consecutive cross-attention based feature layers adaptively exchange spatio-contextual information radar, leading robust attentive achieves state-of-the-art 41.1% mAP 52.3% NDS on nuScenes test set, which is 8.7 10.8 higher than camera-only baseline, well yielding competitive LiDAR method.